Spatial light modulator apparatus

ABSTRACT

A spatial light modulator apparatus for an optical microscope system including an optical head with a mounting flange for mounting the optical head to a port of the microscope, a DMD for generating a pattern image of light, a light source mount for receiving a source of illumination, and one or more optical elements for directing light from the source of illumination to the DMD and to direct the pattern image generated by the DMD to the microscope. A DMD controller has a digital input and is connected to the DMD for driving the individual micromirrors of the DMD to generate the pattern image. A pattern generation subsystem is configured to output pattern image data and a digital interface is connected between the digital input of the DMD controller and the pattern generation subsystem, the digital interface configured to provide a digital drive signal to the DMD controller corresponding to the pattern image data.

RELATED APPLICATIONS

This application is a divisional patent application of U.S. patentapplication Ser. No. 10/991,256 filed on Nov. 17, 2004, which is adivisional patent application of U.S. patent application Ser. No.10/191,947 filed on Jul. 9, 2002, which claims benefit of and priorityto U.S. Provisional Patent Application Ser. No. 60/337,801 (now U.S.Pat. No. 6,885,492) filed Nov. 8, 2001 entitled AN ILLUMINATION SYSTEMFOR MICROSCOPES WHICH EMPLOYS A SPATIAL LIGHT MODULATOR DEVICE.

FIELD OF THE INVENTION

This invention relates to a spatial light modulator apparatus and systemfor use in conjunction with an optical microscope.

BACKGROUND OF THE INVENTION

Spatial light modulators such as liquid crystal displays (LCDs) anddigital mirror devices (DMDs) (as available from Texas Instruments, forexample) are used to modulate incident light into a spatial pattern toform a light image corresponding to an electrical or optical input. DMDshave been successfully incorporated into video projectors and printers,for example. See U.S. Pat. No. 5,535,047 incorporated herein by thisreference.

In the field of microscopes, it is often desirable to create a maskpattern to vary the illumination or viewing properties of the microscopeusing techniques such as transmissive illumination, incidentillumination, dark field illumination, bright field illumination,oblique illumination, differentially shaded illumination, phase contrastillumination, differential polarization illumination, and the like. Inone example, a technique known as fluorescence recovery afterphotobleaching (FRAP) involves labeling specific proteins within aliving cell with fluorescent dyes and then selected areas areirreversibly photobleached by an intense flash of light and thediffusional mobility of the protein is measured by measuring thefluorescence recovery through the exchange of bleached for non-bleachedprotein. In this example, a small area of the slide containing the cellsis targeted and measured comparatively to the surrounding structure. Inthis procedure, the ability to control the spatial distribution of theillumination of the microscope for targeting and measurement iscritical.

In the past and even today these techniques were accomplished usingmechanical pinholes or irises to form the mask pattern image. See U.S.Pat. No. 4,561,731 also incorporated herein by this reference.

After the advent of spatial light modulators such as LCDs and DMDs,however, those skilled in the art soon began proposing these types ofmodulators in microscope systems instead of pinholes or irises to formmask patterns. A DMD is shown in U.S. Pat. No. 5,535,047 incorporatedherein by this reference.

Surprisingly, however, the art is currently limited a) to speciallyconfigured microscopes employing LCDs and DMDs, (see, for example, U.S.Pat. Nos. 5,923,036; 5,587,832; and 5,923,466 incorporated herein bythis reference) or, alternatively, b) to a microscope coupled to acomplete video projector—the video projector itself incorporating, interalia, an LCD or DMD, a light source, and the associated driver. See U.S.Pat. No. 6,243,197 incorporated herein by this reference.

The drawbacks to such configurations are many. Specially designedmicroscopes are expensive and render obsolete the user's existingmicroscopes. Video projector type illuminating devices coupled to anexisting microscope, on the other hand, results in an unduly complex,bulky, and expensive design, and, moreover, results in flickers andrestrictions on the range of wavelengths which can be used in themicroscope due to, inter alia, both the video projector design and thevideo input signal which operates the video projector.

For example, the use of a commercially available video projector with aDMD controlled by a computer graphics card for illumination in amicroscope (U.S. Pat. No. 6,243,197) has significant drawbacks. Sincevideo projectors typically use a rotating color wheel between the lightsource and the DMD, the micromirrors must be synchronized to the primarycolor segments of the spinning wheel. This type of illuminationinherently ‘flickers’ and, although acceptable for overhead projections,is not adequate for scientific microscopy. The temporal switchinginherent in the method of pulse-width modulation (PWM) used to varyintensity levels in a projector system, combined with the electricalnoise of the computer graphic card interface, may result in unacceptableflicker in a microscope system. Furthermore, the spectral nature of thelight source and optical coatings used in video projectors wouldrestrict microscope studies to a wavelength range narrower than usual.

What is needed is an optical head which can be easily coupled to a widevariety of existing microscopes and employing DMD or other spatial lightmodulation technology to digitally generate mask patterns in real timeand without the necessity of the other components and the limitationsassociated with video projectors. Such a spatial light modulatorapparatus is useful, for example, when carrying out FRAP and othertechniques and procedures.

SUMMARY OF THE INVENTION

It is therefore an object of this invention to provide a spatial lightmodulator apparatus which is less complex than a complete videoprojector unit with the necessary modification and additions, which islower in cost, and which performs better.

It is a further object of this invention to provide such an apparatuswhich is useful with and easily coupled to existing microscopes.

It is a further object of this invention to provide such an apparatusand indeed even a complete system which typically requires little or nomodification of the user's existing microscope.

It is a further object of this invention to provide such an apparatusand system which can be easily tailored to the user's specificmicroscope.

It is a further object of this invention to provide such an apparatuswhich employs the high pixel density and wide spectral bandwidth ofcurrent and future DMD technologies without the need for a separateprojector.

It is a further object of this invention to provide such an apparatuswhich interfaces directly with the DMD.

It is a further object of this invention to provide such an apparatuswhich eliminates the flicker associated with video projectors.

The invention results from the realization that instead of designingspecially configured microscopes or coupling a complete video projectorto an existing microscope, a lower cost spatial light modulator whichproduces mask pattern images in real time is effected by the combinationof a specially designed optical head itself incorporating a DMD and aDMD controller digitally controlled by a digital interface responsive toa computerized pattern generation subsystem to provide a digital drivesignal to the DMD controller corresponding to pattern image dataproduced by the pattern generation subsystem.

Instead of a video projector responsive to an analog VGA signal outputfrom a computer graphics card as set forth in the '197 patent, thepresent invention operates solely in the digital realm by the use of aDMD interface which, in essence, provides to the DMD controller a map ofDMD mirror settings corresponding to the mask pattern desired by theuser. In this way, the illumination lamp need not be integral with themodulator apparatus and instead a wide variety of light source inputsmay be used. Moreover, the complexity and cost of the modulatorapparatus is severely reduced, the DMD controller is now a simplebuffer, the modulator apparatus can be mounted to either the reflectedlight port, the transmitted light port, or the camera port of themicroscope, a laser light input can even be used, and real time imagingaccomplished.

This invention features a spatial light modulator apparatus for anoptical microscope system, the apparatus comprising an optical headincluding a mounting flange for mounting the optical head to a port ofthe microscope, a spatial light modulator device such as a DMD forgenerating a pattern image of light, a light source mount for receivinga source of illumination, and one or more optical elements for directinglight from the source of illumination to the DMD and to direct thepattern image generated by the DMD to the microscope. A DMD controllerhas a digital input and is connected to the DMD for driving theindividual micromirrors of the DMD to generate the pattern image. Apattern generation subsystem is configured to output pattern image dataand a digital interface is connected between the digital input of theDMD controller and the pattern generation subsystem and configured toprovide a digital drive signal to the DMD controller corresponding tothe pattern image data.

Preferably, the DMD controller is mounted on the optical head adjacentthe DMD and the digital interface is housed on a PC card received in acomputer and the pattern generation subsystem is operable on thecomputer.

In one example, the optical head includes a housing with an illuminationaxis, the DMD is located on the housing on the illumination axis thereofand the mounting flange is located on the housing on the illuminationaxis thereof opposite the DMD. The light source mount is typicallylocated on the optical head housing on an axis transverse to theillumination axis. In the same example, the optical head furtherincludes a light baffle positioned such that when the micromirrors ofthe DMD are in the off state, light reflected by the DMD is directed tothe light baffle. The optical microscope has a field plane and the DMDand the optical elements of the optical head may be configured to directthe pattern image generated by the DMD to the field plane of the opticalmicroscope. Alternatively, the DMD and the optical elements of theoptical head are configured to direct the pattern image generated by theDMD to a conjugate of the field plane of the optical microscope.

A camera is typically attached to the microscope, the camera having afield of view and the optical elements of the optical head areconfigured such that the DMD pattern image fills part or all of thefield of view of the camera.

A light source is typically disposed in the light source mount such as ahalogen lamp, a flash lamp, an arc lamp, or a laser light source. In thepreferred embodiment, the light source mount further includes a shutterassembly and a filter holder assembly. The optical head may furtherinclude a secondary light source mount also with a shutter assembly anda filter holder assembly. The optical head may further include a beamsplitter mount.

In the preferred embodiment, the digital interface includes a clockwhich provides a clock signal and a logic device responsive to thepattern image data and the clock signal and configured to assign pixelsto the pattern image data and to serialize the assigned pixels accordingto the clock signal to reformat the pattern image data to correspond tothe spatial addressing of the DMD and is further configured to generatea plurality of timing signals based on the clock signal to synchronizeserialization of the assigned pixels. The plurality of timing signalstypically correspond to DMD address counter signals. Several of theplurality of timing signal may be multiplied in frequency by the logicdevice to provide faster global dark resets of the mirrors of the DMD.In the preferred embodiment, the logic device is a field programmablegate array and a digital cable connects the digital interface to thedigital input of the DMD controller. The DMD controller is configured tobuffer the reformatted mask image data and to load the memory cells ofthe DMD. The DMD controller is further configured to provide a resetcommand, a new state command, and a hold command to reset, new state, orhold, respectively, all of the mirrors of the DMD simultaneously.

In one example, the optical microscope includes a camera for imaging thespecimen viewed by the microscope live on a display. The patterngeneration subsystem is configured to output pattern image data andcomprises a drawing editor responsive to an input device for drawing apattern shape and an alpha blending routine responsive to the camera andthe drawing editor for representing the drawn pattern shapetranslucently on the display over the specimen image. The digitalinterface is responsive to the drawing editor to provide a digital drivesignal which controls the DMD to generate the pattern image which isidentical in shape to the translucent pattern shown on the display. Thepattern generation subsystem typically further includes a set of storedcalibration values and a spatial scale and offset routine interposedbetween the drawing editor and the digital interface and responsive tothe stored calibration values for correlating the pixels of the drawnpattern shape to the pixels of the DMD.

This invention also features an optical head for a spatial lightmodulator system, the optical head comprising a mounting flange formounting the optical head to a port of the microscope, a spatial lightmodulator device for generating a pattern image of light, a light sourcemount for receiving a source of illumination, one or more opticalelements for directing light from the source of illumination to thespatial light modulator device and to direct the pattern image generatedby the spatial light modulator device to a microscope, and a controllerconnected to the spatial light modulator device for driving the spatiallight modulator device to generate the pattern image, the controllerhaving a digital input responsive to a digital drive signal. Typically,the spatial light modulator device is a DMD located on the housing onthe illumination axis thereof and the mounting flange is located on thehousing on the illumination axis opposite the DMD.

This invention also features a digital interface interconnected betweena controller for a spatial light modulator device (e.g. a DMD) and apattern generation subsystem, the digital interface comprising a clockwhich provides a clock signal and a logic device responsive to patternimage data output by the pattern generation subsystem and the clocksignal and configured to assign pixels to the pattern image data and toserialize the assigned pixels according to the clock signal to reformatthe pattern image data to correspond to the spatial addressing of thespatial light modulator device and the logic device is furtherconfigured to generate a plurality of timing signals based on the clocksignal to synchronize serialization of the assigned pixels. Theplurality of timing signals usually correspond to DMD address countersignals and several of the said DMD address counter signals arepreferably multiplied in frequency by the logic device to provide fasterglobal dark resets. Typically, the logic device is a programmable logicdevice or a field programmable gate array and further included is adigital cable connecting the digital interface to a DMD controllerconnected to the DMD. The DMD controller is preferably configured tobuffer the reformatted pattern image data and to load the memory cellsof the DMD. The logic device may be further configured to provide areset command, a new state command, and a hold command. Preferably, thedigital interface is on a PC card received in a computer and wherein thepattern generation subsystem is operable on the computer.

This invention also features a pattern generation subsystem configuredto output a pattern image data to a spatial light modulator (e.g., aDMD), the pattern generation subsystem comprising a drawing editorresponsive to an input device for drawing a pattern shape and an alphablending routine responsive to a camera and the drawing editor forrepresenting the drawn pattern shape translucently on a display over aspecimen image. A digital interface may be included responsive to thedrawing editor to provide a digital drive signal which controls thespatial light modulator to generate the pattern image. Further includedis a set of stored calibration values and a spatial scale and offsetroutine interposed between the drawing editor and the digital interfaceand responsive to the stored calibration values for correlating thepixels of the drawn pattern shape to the pixels of the spatial lightmodulator.

In a preferred embodiment, a spatial light modulator system for anoptical microscope, in accordance with this invention, features anoptical head including a mounting flange for mounting the optical headto either the reflected light port or the transmitted light port of themicroscope, a spatial light modulator for generating a pattern image oflight, a light source mount for receiving a source of illumination, andone or more optical elements for directing light from the source ofillumination to the spatial light modulator and to direct the patternimage generated by the spatial light modulator to the microscope. Acontroller is mounted on the optical head adjacent the spatial lightmodulator and has a digital input. A computer includes a patterngeneration subsystem configured to output pattern image data and adigital interface PC card configured to provide a digital drive signalto the controller corresponding to the pattern image data generated bythe pattern generation subsystem.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, features and advantages will occur to those skilled inthe art from the following description of a preferred embodiment and theaccompanying drawings, in which:

FIG. 1 is schematic view of a prior art lighting device for a microscopeemploying a complete video projector as set forth in U.S. Pat. No.6,243,197;

FIG. 2 is a schematic view showing the primary components associatedwith one embodiment of the spatial light modulator apparatus of thesubject invention and, in particular, the optical head thereof;

FIG. 3 is a schematic top side view of another embodiment of the opticalhead of the subject invention;

FIG. 4 is a schematic rear view of the optical head shown in FIG. 3;

FIG. 5 is a schematic bottom side view of the optical head shown inFIGS. 3 and 4;

FIG. 6 is a schematic view showing the optical head of FIGS. 3-5 coupledto a microscope and a laser light source coupled to the optical head inaccordance with the subject invention;

FIG. 7 is a block diagram showing the primary components associated witha complete spatial light modulator apparatus in accordance with thesubject invention;

FIG. 8 is a block diagram showing the primary components associated withthe digital interface of the apparatus shown in FIG. 7;

FIG. 9 is a functional flow chart depicting the operation of the digitalinterface shown in FIG. 8;

FIG. 10 is a block diagram showing the primary components associatedwith the DMD controller of this invention; and

FIG. 11 is a diagram showing the interrelation between the variouscomputer software and hardware components of a complete system inaccordance with the subject invention.

DISCLOSURE OF THE PREFERRED EMBODIMENT

Aside from the preferred embodiment or embodiments disclosed below, thisinvention is capable of other embodiments and of being practiced orbeing carried out in various ways. Thus, it is to be understood that theinvention is not limited in its application to the details ofconstruction and the arrangements of components set forth in thefollowing description or illustrated in the drawings.

Prior art lighting device 10, FIG. 1 for microscope 12 is a completevideo projector unit purporting to employ LCD or DMD 14, light source15, and drive 23 connected via video cable 16 to graphic card 18 of“control/calculating device” 20. Device 10 is connected to transmittedlight port 22 of microscope 12 and graphics card 18 generates the imagesignal for driving LCD/DMD 14. The other components of this system aredescribed in U.S. Pat. No. 6,243,197. The drawbacks, limitations, andproblems associated with such a system are discussed in the backgroundsection above.

In this invention, the spatial light modulator apparatus, in oneembodiment, includes optical head 30, FIG. 2 with housing 32 havingmounting flange 34 thereon for mounting optical head 30 to the reflectedlight port 36 of microscope 38. Optical head 30 may alternatively bemounted to port 40 instead of transmitted light source 41, or evencamera port 42 typically employing camera 56.

Light source mount 44 receives a source of illumination 46 such as ahalogen lamp, a flash lamp, an arc lamp, or even a laser light source.Light from illuminated light source 46 is directed to a spatial lightmodulator device, preferably DMD 48 via optical element 58 (e.g., amirror) and the pattern generated by DMD 48 is directed via opticalelements 52, 54 (e.g., refractive or reflective optical elements or acombination of both) to microscope 38. Light baffle 51 is positioned asshown such that when the micromirrors of DMD 48 are in the off state,light reflected by DMD 48 is directed to light baffle 51.

Thus, optical head 30 contains the optical subsystem which serves tooptically relay the image generated by DMD 48 to field plane 37 or to aconjugate of field plane 37 of microscope 38. Since field plane 37 isconjugate to specimen plane 39, the image of the surface of DMD 48 isprojected onto specimen plane 39. Each spatial position on the surfaceof DMD 48 is represented by a spatial position on specimen 45 located atspecimen plane 39. The size of that representation is dependent upon themagnification factors of the optical subsystem including opticalelements 52 and 54 and the magnification factors of the reflected lightport 36 and objective 43 of microscope 38. The magnification factors inthe reflected light and transmitted light portions of microscope 38differ between different microscope models. Thus, the optical subsystemincluding optical elements 52 and 54 may be adjustable such that theimage size of DMD 48 remains within usable limits. If the DMD imagegreatly overfills the entire field of view, the spaces between themicromirrors may become apparent to the user and illumination ofspecimen 45 may have a lattice-like appearance. If the DMD imagesignificantly underfills the field of view, the benefits of the subjectinvention may likewise not be realized. Thus, ideally, the DMD imageshould fill the entire field of view of camera 56.

In the particular embodiment shown, housing 32 defines illumination axis60 and DMD 48 is located on housing 32 on illumination axis 60 andmounting flange 34 is located on housing 32 opposite DMD 48 but also onillumination axis 60. Light source mount 44 is located on housing 32 onan axis which intersects illumination axis 60 at DMD 48.

In the prototype design shown in FIGS. 3-5, DMD controller 80, FIG. 4 ismounted on optical head 30 adjacent to DMD 48 and further included isshutter assembly 82 and filter holder assembly 84, FIG. 3 for lightsource mount 44. Shown in FIG. 3 is secondary light source mount 86 withshutter assembly 88 and filter holder assembly 90, and beam splittermount 92. See U.S. Pat. No. 5,933,274 incorporated herein by thisreference.

FIG. 6 is a block diagram of the embodiment shown in FIGS. 3-5 mountedon microscope 38 and further showing a laser illumination sourceattached to light source mount 44. Beam splitter mount 92 allows theplacement of an optical element which permits the mask pattern imagegenerated by DMD 48 to pass through and into microscope 38 while alsoreceiving illumination from a secondary light source attached tosecondary light source mount 86 and directing that illumination intomicroscope 38 in a manner coaxial with the mask pattern generated by DMD48. Thus, specimen 45 on microscope 38 can be illuminated overall in aconventional manner with one light source and simultaneously illuminatedwith a mask pattern.

A complete spatial light modulator apparatus in accordance with thisinvention also features DMD controller 80, FIG. 7 having digital inputconnector 100 connected to digital interface 104 of computer 106 viadigital interface cable 102. Pattern generation subsystem 103 alsooperating on computer 106 and discussed in further detail below isconfigured to output pattern image data to digital interface 104 whichprovides a digital drive signal to DMD controller 80 via digital cable102 corresponding to the pattern image data. DMD controller 80, in turnis connected to DMD 48 and drives the individual micromirrors of DMD 48to generate the pattern image transmitted via optics 52 and 54, FIGS. 2and 6, to microscope 38.

In one embodiment, digital interface 104, FIG. 7 is housed on PC card120, FIG. 8 received in computer 106, FIG. 7 and has digital outputconnector 105 connected to digital cable 102, FIG. 7. As shown in FIG.8, digital interface 104 includes clock 122 which provides a clocksignal to logic device 124 (e.g., a field programmable gate array(FPGA)). Logic device 124 is responsive to pattern image data providedby the pattern generation subsystem 103 via computer bus interface 180and PC Card connection 181 and the clock signal and is configured toassign pixels to the pattern image data and to serialize the assignedpixels according to the clock signal to reformat the pattern image datato correspond to the spatial addressing of DMD 48. DMD controller 80,FIG. 7 is configured to buffer the reformatted mask image data and toload the static-RAM (SRAM) memory cells of DMD 48. Logic device 124,FIG. 8 of digital interface 104 is also configured to generate aplurality of timing signals based on the clock signal to synchronizeserialization of the assigned pixels. In the preferred embodiment, theplurality of timing signals correspond to DMD address counter signalsbut may be multiplied in frequency by logic device 124 to provide fasterglobal dark resets of the mirrors of DMD 48, FIG. 7. Logic device 124,FIG. 8 is preferably configured to provide a reset mirror command,switch to a new mirror state command, and a hold mirror state command toreset, new state, or hold all of the mirrors of DMD 48, FIG. 7simultaneously.

Referring again now to FIG. 7, pattern generation subsystem 103, namelysoftware operating on computer 106, is configured to output patternimage data to digital interface 104 which provides the digital drivesignal on digital cable 102 to DMD controller 80 corresponding to thepattern image data. Pattern generation subsystem 103 provides abitmapped mask image 170, FIG. 9 containing an informational modeheader. Camera 56, FIG. 7 which is mounted on the microscope isconnected via camera cable 150 to camera interface 152 located withincomputer 106. Camera interface 152 may be a frame grabber type card tointerface to analog camera video signals, to digitize those signals intovideo data, and to distribute that data into the computer data bus. Inanother embodiment, camera interface 152 is a digital device such as anIEEE 1394 interface to allow use of cameras compliant with the IEEE 1394digital communications standard.

Typically, each DMD 48 contains an array of several hundred thousandmicromirrors on the surface. Each micromirror is typically 16 micronssquare and separated from each other by a 1 micron space. Eachmicromirror is addressable electronically. With a logic state of high inthe SRAM beneath the micromirror, the micromirror tilts to a positive10° position, typically, which herein is called the “on” state. With alogic state of low, the micromirror tilts to a negative 10° position,typically, which herein is called the “off” state. The light emittedfrom light source 46, FIG. 2 which is incident on the off statemicromirrors of DMD 48 is reflected along an axis other thanillumination axis 60 into light baffle 51. In the case of the “on” statemicromirrors, light emitted from light source 46 is reflected from thesemicromirrors of DMD 48 along illumination axis 60 into microscope 38.The light is then directed to specimen plane 39 by microscope 38,illuminating specimen 45 in a pattern matching the “on” micromirrors ofDMD 48. The size of the illuminated pattern is dependent on the opticalmagnification factors mentioned above.

For the purposes of illustration, assume that specimen 45 is somewhatreflective, light source 46 is turned on and the micromirrors of DMD 48representing the pattern “X” are in the “on” state. The image at eyepoint 63 is the image of specimen 45 superimposed on the image of themicromirror pattern “X” of DMD 48. The same image is present at camera56 and at computer monitor 154, FIG. 7. If objective 43, FIG. 2 ofmicroscope 38 is changed to one of a higher magnification, the image ofspecimen 45 at the same locations will be larger but the pattern “X”image of the array of micromirrors of DMD 48 will not change in size ateye point 63.

In the art, the term “pixel” is used in a variety of contexts. Forexample, the manufacturer of DMD 48 calls each micromirror a pixel, DMD48 is said to break the light into a plurality of separate regionscalled pixels, the digital video image of specimen 45 under microscope38 is composed of pixels, and monitor 154, FIG. 7 is resolved intopixels. To avoid confusion, the following nomenclature is used herein.The spatial representation of each micromirror of DMD 48, FIGS. 2 and 7is called a DMD pixel, and the spatial representation of the imageacquired by video camera 56 and viewed on computer monitor 154 is calleda screen pixel.

Using keyboard 156 and computer mouse 158 in conjunction with the customsoftware 103 of this invention, the user is able to control specimenillumination. Software 103 controls DMD 48 by way of digital interface104, digital interface cable 102 and DMD controller 80.

Software 103, FIG. 11 displays on monitor 154 a live video image of thespecimen captured by camera 56 transmitted over camera cable 150 tocamera interface 152. The live video image of the specimen is displayedon monitor 154 for reference only in order to target what areas are tobe masked. By using mouse 158 and the drawing editor 160 portion ofsoftware 103, the user has the ability to draw translucent mask-overlaysof any desired shape which superimpose upon the image of the specimen.The user will still be able to see the live video image underneath thetranslucent mask overlay. The user will also be able to select from aset of pre-defined translucent mask-overlays (including full-field) orto select and recall user-defined translucent mask-overlays from storedcomputer files. The user can than select to have the specimenilluminated in a pattern identical to the shape of these overlays.

Drawing editor 160 of pattern generation subsystem 103, FIG. 11 isresponsive to keyboard 156, FIG. 7 or mouse 158 or any other inputdevice for drawing a pattern shape. Alpha blending routine 162, FIG. 11,is responsive to camera 56, FIG. 7 (via camera interface 152) anddrawing editor 160, FIG. 11 for representing the pattern drawn usingdrawing editor 160 translucently on the display or monitor 154 over thespecimen image acquired by camera 56. Thus, digital interface 104, FIG.7 is responsive to drawing editor 160, FIG. 11 to provide a digitaldrive signal which controls DMD 48, FIG. 7 via DMD controller 80 togenerate a pattern mask image which represents the translucent patternshape. Pattern generation subsystem 103 may further include a set ofstored calibration values 166, FIG. 11 and a spatial scale and offsetroutine 168 interposed between drawing editor 160 and digital interface104 to be responsive to the stored calibration values 166 forcorrelating the screen pixels of the drawn pattern shape to the pixelsof DMD 48.

More specifically, drawing editor 160, FIG. 11 preferably generates datafor the mask-overlay image to be displayed on monitor 154 and alsocombines that data with calibrated spatial scale and offset informationvia routine 168 and the calibration values stored at 166 to form a solidmask image to be transferred to DMD 48. The three images are separatesets of data, the specimen image is solid, the mask-overlay image istranslucent and is overlayed upon the specimen image and thirdly, themask image is solid, scaled, and offset.

Scale and offset routine 168 in connection with calibration values 166,allows scaling and offset in two spatial axes, in order to achieveaccurate and repeatable registration between screen pixels in thetranslucent mask-overlay and the DMD pixels. DMD pixels correspondoptically to the spatial positions at specimen plane 39, FIG. 6 whichcorrespond to camera pixels which correspond back to screen pixels onthe live video image on monitor 154, FIG. 11. In this way, targeting ofthe desired areas may be done in advance of the illumination sequencewith accuracy. Calibration is not required on a frequent basis, buttypically preformed only at the time of set-up of the system or uponchanging optical subsystem 52, 54, FIG. 2. Drawing editor 160, FIG. 11data for the mask-overlay image are sent through alpha blending routine162 to produce desired translucently and then displayed on monitor 154.Drawing editor 160 data is also sent through scale and offset routine168 and then directed to digital interface 104 over the data bus of thecomputer.

As stated above, digital interface 104 incorporates logic device 124,FIG. 8 such as a field programmable gate array (FPGA). Digital interface104 reformats the mask image data to correspond with the spatialaddressing of DMD 48, FIG. 11, creates timing and control sequences andprovides the reformatted data and sequences to DMD controller 80 viadigital interface cable 102. DMD controller 80, FIG. 10 buffers the maskimage data and sequences, loads the memory cells of DMD 48, and providesadditional control signals and voltages required by DMD 48.

Software 103, FIG. 7 allows the user to control the timing of theillumination in a format such as with pull-down menus. Predetermined anduser-defined timing schedules are available to the user. Automatic,manual, and externally triggered modes of operation are possible.

Thus, drawing editor 160, FIG. 11 is used to draw a mask image, andscale and offset routine 168 processes that image according to thestored calibration values 166. This output to digital interface 104 isin the form of a bitmap image and added to the image may be aninstructional mode header from software control module 165, FIG. 11 todesignate the operational mode (single mask image, multiple repetitionsof the image, “live” mode where the mask is generated as fast aspossible while the user manipulates drawing editor 160) and informationabout how the mask image is to be produced such as start time, durationof mask (on time) internal or external trigger, and/or trigger outselection (for triggering the user's light source or detector). Thisblock of data may be transferred over the computer PCI bus by a standardtechnique called direct memory access (DMA) 190, FIG. 9 to digitalinterface 104.

Alternatively, the mask image could be in the form of vectorized datainstead of bitmap data. For instance the coordinates for the corners ofa shape such as a square would transfer faster over DMA. Then, however,logic device 124 of digital interface 104 would have the additional taskof bitmapping the data to correspond with the DMD array and “OR”-ingthat data to the RAM memory to over-write only the areas changed fromthe last frame. The serialization of data by the FPGA for output to theDMD controller would remain the same.

Digital interface 104, FIGS. 8 and 9 thus preferably has PC interface180 which listens to the computer bus. The block of data flows throughPC interface 180 and through logic device 124. Logic device 124functions to strip off and retain the informational mode header,allowing the bitmap data to flow into random access memory (RAM) 182 asshown in FIG. 9. Logic device 124 then uses clock 122 to generate theappropriate counter reset signals (timing generation) and synchronouslyreads bitmap data out of RAM 182 in a serial fashion; for example, 9600bits long×50 channels wide, consistent with the structure of the DMDaddressing scheme. These data along with the timing signals are sent outover digital interface cable 102, FIG. 7 through DMD controller 80 andinto the SRAM addresses on DMD 48. When this is done, according to theoperation mode set (trigger, duration, and the like) logic device 124,FIG. 8 issues a “Mirror drive command” which, when interpreted by DMDcontroller 80, FIG. 10 causes the micromirrors of the DMD 48 to reset,then read their new states from the DMD SRAM addresses, tilt to thosenew states, and hold until the next such command.

At some time specified by the operator, through external trigger orsoftware (duration), according to the mode, the DMD will be requested togo dark. The quicker the DMD can go dark again after displaying a mask,the better the performance in a situation requiring a brief pulse oflight. In the conventional manner of controlling a DMD device, the“Dark” is a global command line wired to the DMD through the DMDcontroller. The Dark command essentially writes logic state low to allof the RAM addresses on the DMD during a sequence of timing signalsregardless of data input states. Since the addresses are selected by thetiming signals, the effect is not instantaneous but normally distributedover 640 clock cycles. In the system of the present invention, logicdevice 124 will issue a Dark sequence, logic device 124 will hold theDark command high, and further generate specially compacted timingsignals (compacted by a factor of four) to allow global addressing ofthe Dark command four times faster than normal. The logic device neednot disturb the bitmap image in RAM memory 182 from the last mask image.For repetitive mode, logic device 124 functions as described above. Forlive mode, logic device 124 will not enter the Dark sequence, butcommunicates with the software over bus interface 180 to indicate thatan image has been sent to DMD controller 80, FIG. 7 and a Mirror drivecommand issued. This command instructs the software to begin anothersequence of data block (bitmap image and header) transferred via DMA todigital interface 120, FIG. 8. In this way, when the operator is drawingor moving a drawn shape with drawing editor 160, FIG. 11, the mask willtrack those movements as fast as the system will allow with no darkframes in between.

DMD controller 80, FIG. 10 is connected to digital interface cable 102as discussed above and appropriately buffers the pattern image data 200as shown at 210 along with the timing signals 202 and Dark signal andsends them to DMD 48. It also converts the Mirror drive command signalsto the high voltage analog waveform as shown at 208 required toelectrostatically tilt the mirrors of DMD 48. Although DMD 48 is furtherdivided into fifteen groups of rows, each with its Mirror drive commandline, all of these lines are actuated simultaneously for evenillumination. Note that Texas Instruments, the manufacturer of thepreferred DMD 48, activates these groups in sequence to take advantageof the writing of each column from top to bottom. The row group iswritten to, and then, while the next row group is being written to, theMirror drive command for the first group is issued. This, however,results in the image being drawn in blocks from top to bottom whereas inthe system of this invention the image appears all at once.

DMD controller 80 also contains various voltage regulation andconversion function 206 to power DMD 48 and to select DMD 48 attributesby maintaining various DMD pins at certain voltages as shown at 212.

The benefits of the present invention over the prior art are several andsignificant. The most evident advantage achieved is the ability tocontrol and target the spatial distribution of illumination in aconventional microscope. Areas of the specimen to be illuminated can bevery small—on the order of few microns—or the areas can be very large—onthe order of hundreds of microns. Another major advantage of the presentinvention is the ability to customize the pattern of illumination whichwill be transmitted to the specimen in the microscope. This feature isof great benefit to researchers in the life sciences industry who needto illuminate or mask biological structures which are seldom geometricand uniform. By virtue of the ability to illuminate a specimenselectively, a researcher will be able to gain the ability to monitorand measure by photonic means select areas of the specimen with the samelevel of control.

Utilizing the digital mirror device which exhibits high speed, highpixel density, and a wide spectral bandwidth extending down into thenear ultraviolet which is desirable in microscopy, provides an equallyevident advantage in the ability to control the timing of illuminationin a conventional microscope. Areas of the specimen can be illuminatedin an infinite combination of repetitions and durations, ranging frompulses of a few hundreds of microseconds duration to continuousillumination. Conventional mechanical shutters can be installed into theillumination path of the microscope, but they are slower than the systemof the present invention. Beam shape, for example, is not controllablewith mechanical shutters alone because the aperture opening is typicallypredetermined. The present invention communicates directly with the DMDin a true black and white temporally-static manner unlike DMD projectorswith standard pulse width modulated DMD interfaces. This eliminates anyraster-type scanning of the mask pixels which reduce optical throughputand might adversely effect sensitive scientific experiments. The abilityto customize the illumination of a microscope in real time using atranslucent live-video interface is unique to the subject invention.

Thus, the present invention provides an illumination system for aconventional microscope which gives the user both spatial and temporalcontrol over specimen illumination. The array of programmablemicromirrors is inserted in the illumination axis such that whenactivated they allow light to be reflected along the illumination axisto the specimen plane. Each micromirror directs light toward a spatialposition of the specimen plane which corresponds to the spatial positionof the micromirror in a conjugate plane. The innovative softwaredeveloped and described above enables the user to select areas in thefield of view to be illuminated, or conversely to be masked and notilluminated by a light source. The user is also able to control thetiming of illumination, ranging from single pulses of varying length, tocontinuous illumination.

As an example, a halogen light source 41, FIG. 2 may be used totransluminate from below a culture of living cells on the stage of aconventional microscope. Video camera 56 is mounted to the camera portof microscope 38 with the data output connected to the camera interface152 of computer 106, FIG. 7 which displays the live-video image of thespecimen on computer monitor 154. The researcher desires to illuminate,i.e., target, some of the cells in the field of view with a second lightsource and not other cells. Using computer mouse 158 and software 103 ofthis invention, the user draws translucent overlays on computer monitor154 superimposed on but not obscuring the target cells. The user thenselects to have these targets illuminated by a light source 46 such asan arc lamp used for epifluorescence excitation which is made toilluminate DMD 48 by way of optical element 58. Patterned light will bedirected into microscope 38 along illumination axis 60 and will shinedown from above onto the specimen culture at specimen plane 39. Thespatial position of the mask-overlays relative to the specimen imagecorrespond to the spatial position of the micromirrors of the DMD. Theuser then selects a time sequence and triggers the software. A spatialmap of the overlays is encoded and directed to the DMD which reflectsthe light from light source 46 to the cell culture for a selected periodof time. Thus, the researcher now has the ability to uniquely perform aphotonic experiment on one part of a culture sample while simultaneouslyperforming the control experiment or additional experiments on relatedparts of the same sample.

Although specific features of the invention are shown in some drawingsand not in others, this is for convenience only as each feature may becombined with any or all of the other features in accordance with theinvention. The words “including”, “comprising”, “having”, and “with” asused herein are to be interpreted broadly and comprehensively and arenot limited to any physical interconnection. Moreover, any embodimentsdisclosed in the subject application are not to be taken as the onlypossible embodiments.

Other embodiments will occur to those skilled in the art and are withinthe following claims:

1. A digital interface interconnected between a controller for a spatiallight modulator device and a pattern generation subsystem, the digitalinterface comprising: a clock which provides a clock signal; and a logicdevice responsive to pattern image data output by the pattern generationsubsystem and the clock signal and configured to assign pixels to thepattern image data and to serialize the assigned pixels according to theclock signal to reformat the pattern image data to correspond to thespatial addressing of the spatial light modulator device.
 2. The digitalinterface of claim 1 in which the spatial light modulator is a DMD. 3.The digital interface of claim 1 in which the logic device is aprogrammable logic device or a field programmable gate array.
 4. Thedigital interface of claim 2 further including a digital cableconnecting the digital interface to a DMD controller connected to theDMD.
 5. The digital interface of claim 4 in which the DMD controller isconfigured to buffer the reformatted pattern image data and to load thememory cells of the DMD.
 6. The digital interface of claim 1 in whichthe logic device is further configured to generate a plurality of timingsignals based on the clock signal to synchronize serialization of theassigned pixels.
 7. The digital interface of claim 6 in which theplurality of timing signals correspond to DMD address counter signals.8. The digital interface of claim 7 in which several of the said DMDaddress counter signals are multiplied in frequency by the logic deviceto provide faster global dark resets.
 9. The digital interface of claim8 in which the logic device is further configured to provide a resetcommand, a new state command, and a hold command.
 10. The digitalinterface of claim 1 in which the digital interface is on a PC cardreceived in a computer and wherein the pattern generation subsystem isoperable on the computer.